Orbit: Efficient Processing of Iterations

Various scienti c applications compute iterations on a huge set of input data. Examples include parameter sweep, which processes the same model through hundreds or thousands of input data, scienti c visualization and numeric methods to solve hyperbolic equations. The state of the art for executing such applications is to model them as scienti c work ows and to run them in HPC infrastructure. The execution model strives to combine pipeline parallelism, among activities of the work ow, with intra-work ow parallelism over data partitions, both of them subjected to work ow characteristics. In such scenario, the scienti c work ow execution model can be approximated to that of parallel query processing. In fact, database groups at LNCC and COPPE have implemented parallel work ow engines, QEF and Chiron, under this assumption. In order to full integrate iterations within a data processing execution model, an extension is required to treat the loop control operator. In this paper we investigate the problem of e ciently executing scienti c work ows that include iterations. We introduce Orbit which is a generic operator to manage the data ow in an iterative procedure. An execution model centered into Orbit is proposed including a centralized and parallelized modes. Additionally, two new execution strategies are investigated: rst-tuplerst, rst-iterationrst. We have obtained initial results that evaluate the di erent execution strategies in both centralized and parallel modes.